-
Notifications
You must be signed in to change notification settings - Fork 319
Add dd_tags into djm.tags spark.application span attribute #10004
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Add dd_tags into djm.tags spark.application span attribute #10004
Conversation
6145838 to
c1d1604
Compare
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 12 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~aa411ca865, baseline=1.57.0-SNAPSHOT~c69c2cd420
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1059622
Total [baseline] (10.796 s) : 0, 10795642
Agent [candidate] (1.053 s) : 0, 1053494
Total [candidate] (10.788 s) : 0, 10787714
section appsec
Agent [baseline] (1.227 s) : 0, 1227490
Total [baseline] (10.818 s) : 0, 10817923
Agent [candidate] (1.24 s) : 0, 1240283
Total [candidate] (10.877 s) : 0, 10877016
section iast
Agent [baseline] (1.194 s) : 0, 1194333
Total [baseline] (11.141 s) : 0, 11140853
Agent [candidate] (1.198 s) : 0, 1197761
Total [candidate] (11.218 s) : 0, 11218362
section profiling
Agent [baseline] (1.199 s) : 0, 1199297
Total [baseline] (10.905 s) : 0, 10905060
Agent [candidate] (1.201 s) : 0, 1200917
Total [candidate] (10.951 s) : 0, 10950651
gantt
title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~aa411ca865, baseline=1.57.0-SNAPSHOT~c69c2cd420
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.498 ms) : 0, 1498
crashtracking [candidate] (1.473 ms) : 0, 1473
BytebuddyAgent [baseline] (712.226 ms) : 0, 712226
BytebuddyAgent [candidate] (708.404 ms) : 0, 708404
GlobalTracer [baseline] (250.977 ms) : 0, 250977
GlobalTracer [candidate] (249.404 ms) : 0, 249404
AppSec [baseline] (32.286 ms) : 0, 32286
AppSec [candidate] (32.022 ms) : 0, 32022
Debugger [baseline] (6.533 ms) : 0, 6533
Debugger [candidate] (6.387 ms) : 0, 6387
Remote Config [baseline] (682.797 µs) : 0, 683
Remote Config [candidate] (671.408 µs) : 0, 671
Telemetry [baseline] (14.815 ms) : 0, 14815
Telemetry [candidate] (14.726 ms) : 0, 14726
Flare Poller [baseline] (5.713 ms) : 0, 5713
Flare Poller [candidate] (5.652 ms) : 0, 5652
section appsec
crashtracking [baseline] (1.488 ms) : 0, 1488
crashtracking [candidate] (1.499 ms) : 0, 1499
BytebuddyAgent [baseline] (731.247 ms) : 0, 731247
BytebuddyAgent [candidate] (738.7 ms) : 0, 738700
GlobalTracer [baseline] (240.667 ms) : 0, 240667
GlobalTracer [candidate] (243.736 ms) : 0, 243736
IAST [baseline] (24.859 ms) : 0, 24859
IAST [candidate] (25.381 ms) : 0, 25381
AppSec [baseline] (175.183 ms) : 0, 175183
AppSec [candidate] (176.269 ms) : 0, 176269
Debugger [baseline] (6.236 ms) : 0, 6236
Debugger [candidate] (6.363 ms) : 0, 6363
Remote Config [baseline] (689.69 µs) : 0, 690
Remote Config [candidate] (701.497 µs) : 0, 701
Telemetry [baseline] (8.13 ms) : 0, 8130
Telemetry [candidate] (8.379 ms) : 0, 8379
Flare Poller [baseline] (4.011 ms) : 0, 4011
Flare Poller [candidate] (4.088 ms) : 0, 4088
section iast
crashtracking [baseline] (1.485 ms) : 0, 1485
crashtracking [candidate] (1.482 ms) : 0, 1482
BytebuddyAgent [baseline] (833.621 ms) : 0, 833621
BytebuddyAgent [candidate] (834.46 ms) : 0, 834460
GlobalTracer [baseline] (237.742 ms) : 0, 237742
GlobalTracer [candidate] (239.482 ms) : 0, 239482
IAST [baseline] (32.759 ms) : 0, 32759
IAST [candidate] (34.011 ms) : 0, 34011
AppSec [baseline] (27.683 ms) : 0, 27683
AppSec [candidate] (28.054 ms) : 0, 28054
Debugger [baseline] (6.945 ms) : 0, 6945
Debugger [candidate] (6.069 ms) : 0, 6069
Remote Config [baseline] (609.18 µs) : 0, 609
Remote Config [candidate] (624.834 µs) : 0, 625
Telemetry [baseline] (7.898 ms) : 0, 7898
Telemetry [candidate] (7.986 ms) : 0, 7986
Flare Poller [baseline] (10.787 ms) : 0, 10787
Flare Poller [candidate] (10.754 ms) : 0, 10754
section profiling
crashtracking [baseline] (1.427 ms) : 0, 1427
crashtracking [candidate] (1.44 ms) : 0, 1440
BytebuddyAgent [baseline] (734.638 ms) : 0, 734638
BytebuddyAgent [candidate] (734.905 ms) : 0, 734905
GlobalTracer [baseline] (223.264 ms) : 0, 223264
GlobalTracer [candidate] (222.772 ms) : 0, 222772
AppSec [baseline] (32.149 ms) : 0, 32149
AppSec [candidate] (32.197 ms) : 0, 32197
Debugger [baseline] (7.642 ms) : 0, 7642
Debugger [candidate] (6.887 ms) : 0, 6887
Remote Config [baseline] (708.994 µs) : 0, 709
Remote Config [candidate] (707.744 µs) : 0, 708
Telemetry [baseline] (15.279 ms) : 0, 15279
Telemetry [candidate] (16.243 ms) : 0, 16243
Flare Poller [baseline] (4.157 ms) : 0, 4157
Flare Poller [candidate] (4.124 ms) : 0, 4124
ProfilingAgent [baseline] (110.846 ms) : 0, 110846
ProfilingAgent [candidate] (112.489 ms) : 0, 112489
Profiling [baseline] (111.493 ms) : 0, 111493
Profiling [candidate] (113.146 ms) : 0, 113146
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~aa411ca865, baseline=1.57.0-SNAPSHOT~c69c2cd420
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.05 s) : 0, 1050319
Total [baseline] (8.706 s) : 0, 8706455
Agent [candidate] (1.051 s) : 0, 1050547
Total [candidate] (8.677 s) : 0, 8677263
section iast
Agent [baseline] (1.193 s) : 0, 1193390
Total [baseline] (9.341 s) : 0, 9340909
Agent [candidate] (1.206 s) : 0, 1206434
Total [candidate] (9.375 s) : 0, 9375276
gantt
title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~aa411ca865, baseline=1.57.0-SNAPSHOT~c69c2cd420
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.479 ms) : 0, 1479
crashtracking [candidate] (1.488 ms) : 0, 1488
BytebuddyAgent [baseline] (706.392 ms) : 0, 706392
BytebuddyAgent [candidate] (706.887 ms) : 0, 706887
GlobalTracer [baseline] (248.377 ms) : 0, 248377
GlobalTracer [candidate] (248.428 ms) : 0, 248428
AppSec [baseline] (31.947 ms) : 0, 31947
AppSec [candidate] (31.898 ms) : 0, 31898
Debugger [baseline] (6.395 ms) : 0, 6395
Debugger [candidate] (6.351 ms) : 0, 6351
Remote Config [baseline] (667.426 µs) : 0, 667
Remote Config [candidate] (663.94 µs) : 0, 664
Telemetry [baseline] (14.654 ms) : 0, 14654
Telemetry [candidate] (15.159 ms) : 0, 15159
Flare Poller [baseline] (5.545 ms) : 0, 5545
Flare Poller [candidate] (4.874 ms) : 0, 4874
section iast
crashtracking [baseline] (1.479 ms) : 0, 1479
crashtracking [candidate] (1.494 ms) : 0, 1494
BytebuddyAgent [baseline] (831.413 ms) : 0, 831413
BytebuddyAgent [candidate] (842.524 ms) : 0, 842524
GlobalTracer [baseline] (238.457 ms) : 0, 238457
GlobalTracer [candidate] (239.109 ms) : 0, 239109
IAST [baseline] (31.416 ms) : 0, 31416
IAST [candidate] (30.616 ms) : 0, 30616
AppSec [baseline] (30.492 ms) : 0, 30492
AppSec [candidate] (31.679 ms) : 0, 31679
Debugger [baseline] (6.068 ms) : 0, 6068
Debugger [candidate] (6.151 ms) : 0, 6151
Remote Config [baseline] (620.587 µs) : 0, 621
Remote Config [candidate] (616.679 µs) : 0, 617
Telemetry [baseline] (7.965 ms) : 0, 7965
Telemetry [candidate] (8.137 ms) : 0, 8137
Flare Poller [baseline] (10.631 ms) : 0, 10631
Flare Poller [candidate] (10.952 ms) : 0, 10952
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 3 performance regressions! Performance is the same for 15 metrics, 17 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~aa411ca865, baseline=1.57.0-SNAPSHOT~c69c2cd420
dateFormat X
axisFormat %s
section baseline
no_agent (1.214 ms) : 1202, 1226
. : milestone, 1214,
iast (3.16 ms) : 3121, 3200
. : milestone, 3160,
iast_FULL (5.743 ms) : 5685, 5800
. : milestone, 5743,
iast_GLOBAL (3.674 ms) : 3610, 3738
. : milestone, 3674,
profiling (2.285 ms) : 2263, 2306
. : milestone, 2285,
tracing (1.831 ms) : 1816, 1846
. : milestone, 1831,
section candidate
no_agent (1.215 ms) : 1203, 1227
. : milestone, 1215,
iast (3.344 ms) : 3301, 3387
. : milestone, 3344,
iast_FULL (6.013 ms) : 5952, 6075
. : milestone, 6013,
iast_GLOBAL (3.602 ms) : 3555, 3650
. : milestone, 3602,
profiling (2.165 ms) : 2145, 2185
. : milestone, 2165,
tracing (1.851 ms) : 1834, 1868
. : milestone, 1851,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~aa411ca865, baseline=1.57.0-SNAPSHOT~c69c2cd420
dateFormat X
axisFormat %s
section baseline
no_agent (18.022 ms) : 17839, 18205
. : milestone, 18022,
appsec (18.817 ms) : 18622, 19012
. : milestone, 18817,
code_origins (17.604 ms) : 17429, 17779
. : milestone, 17604,
iast (17.848 ms) : 17670, 18026
. : milestone, 17848,
profiling (19.411 ms) : 19212, 19611
. : milestone, 19411,
tracing (17.998 ms) : 17818, 18177
. : milestone, 17998,
section candidate
no_agent (19.011 ms) : 18816, 19206
. : milestone, 19011,
appsec (18.838 ms) : 18645, 19032
. : milestone, 18838,
code_origins (17.83 ms) : 17652, 18007
. : milestone, 17830,
iast (18.104 ms) : 17922, 18285
. : milestone, 18104,
profiling (18.54 ms) : 18356, 18724
. : milestone, 18540,
tracing (17.814 ms) : 17637, 17991
. : milestone, 17814,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~aa411ca865, baseline=1.57.0-SNAPSHOT~c69c2cd420
dateFormat X
axisFormat %s
section baseline
no_agent (1.487 ms) : 1476, 1499
. : milestone, 1487,
appsec (2.545 ms) : 2490, 2600
. : milestone, 2545,
iast (2.236 ms) : 2171, 2301
. : milestone, 2236,
iast_GLOBAL (2.274 ms) : 2209, 2340
. : milestone, 2274,
profiling (2.1 ms) : 2047, 2153
. : milestone, 2100,
tracing (2.067 ms) : 2016, 2118
. : milestone, 2067,
section candidate
no_agent (1.485 ms) : 1474, 1497
. : milestone, 1485,
appsec (2.545 ms) : 2490, 2600
. : milestone, 2545,
iast (2.231 ms) : 2167, 2296
. : milestone, 2231,
iast_GLOBAL (2.281 ms) : 2216, 2346
. : milestone, 2281,
profiling (2.067 ms) : 2015, 2119
. : milestone, 2067,
tracing (2.059 ms) : 2008, 2110
. : milestone, 2059,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~aa411ca865, baseline=1.57.0-SNAPSHOT~c69c2cd420
dateFormat X
axisFormat %s
section baseline
no_agent (15.662 s) : 15662000, 15662000
. : milestone, 15662000,
appsec (15.007 s) : 15007000, 15007000
. : milestone, 15007000,
iast (18.353 s) : 18353000, 18353000
. : milestone, 18353000,
iast_GLOBAL (17.782 s) : 17782000, 17782000
. : milestone, 17782000,
profiling (15.134 s) : 15134000, 15134000
. : milestone, 15134000,
tracing (14.727 s) : 14727000, 14727000
. : milestone, 14727000,
section candidate
no_agent (15.366 s) : 15366000, 15366000
. : milestone, 15366000,
appsec (14.776 s) : 14776000, 14776000
. : milestone, 14776000,
iast (18.671 s) : 18671000, 18671000
. : milestone, 18671000,
iast_GLOBAL (18.102 s) : 18102000, 18102000
. : milestone, 18102000,
profiling (14.693 s) : 14693000, 14693000
. : milestone, 14693000,
tracing (14.864 s) : 14864000, 14864000
. : milestone, 14864000,
|
7d0d9a5 to
67ee5ad
Compare
67ee5ad to
25323ca
Compare
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
...src/testFixtures/groovy/datadog/trace/instrumentation/spark/AbstractSparkListenerTest.groovy
Show resolved
Hide resolved
25323ca to
aa411ca
Compare
What Does This Do
Puts all
DD_TAGStags intodjm.tagsattribute. This is necessary to identify allDD_TAGSas otherwise it's unclear which span attributes come fromDD_TAGS. Also, the approach is consistent with Airflow and Databricks crawler.Motivation
datadog.running_spans.durationis useful for monitoring job execution time and alerting in case of long running jobs. However, the metric does not allow currently any user provided tagging which which makes group-bys on a metric cumbersome.Additional Notes
Related PR - https://github.com/DataDog/dd-go/pull/210590
Contributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]